Towards Faster Training Algorithms Exploiting Bandit Sampling From Convex to Strongly Convex Conditions

The training process for deep learning and pattern recognition normally involves the use of convex strongly optimization algorithms such as AdaBelief SAdam to handle lots “uninformative” samples that should be ignored, thus incurring extra calculations. To solve this open problem, we propose design bandit sampling method make these focus on “informative” during process. Our contribution is twofold: first, a algorithm with sampling, termed AdaBeliefBS, prove it converges faster than its original version; second, works well algorithms, generalized SAdam, called SAdamBS, SAdam. Finally, conduct series experiments various benchmark datasets verify fast convergence rate our proposed algorithms.